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REMARKS 

Claim 1 was previously amended to correct a typographical error. 
Claims 43-44 were previously withdrawn in r^ponse to an earlier 
restriction requirement. 

Claims 1-42 remain in this application. 

3SU.S,C. 8103 

Claims 1. 6-11, 13-17, and 32^2 

Claims 1, 6-11, 13-17, and 32-42 are rejected under 35 U.S.C. §103(a) as 
being unpatentable over U.S. Patent 6,324,545 to Morag (Morag) in view of U.S. 
Patent 5,572,728 to Tada et al (Tada), Applicants respectfully traverse the 
rejection- 
Amended independent claim 1 recites 

A method comprising: 

identifying, in response to a search query, first multimedia objects 
having an associated keyword that matches a keyword in the search query 
and second multimedia objects that have content features similar to those of 
the first multimedia objects; 

presenting the first and second multimedia objects to a usei^ 

monitoring feedback from the user as to which of the first and 
second multimedia objects are relevant to the search query; and 

annotating one or more of the multimedia objects, which are deemed 
relevant by the user, with the keyword. 

Morag describes creation of personalized picture albums. A customer 
acquires images such as taking pictures with a digital camera* The customer may 
provide a text or voice annotation to the images as they are acquired The 
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customer provides the images, which may include amiotation, to a service 
provider. The service provider allows the customer to provide input (feedback) as 
to how an album of the images arrange prior to a final album being printed and 
sent to the customer. The service provider prints out the final album with the 
pictures (images) and the album is then mailed to the customer. 

Tada describes a system to allow a user to edit important items (multimedia 
objects such as parts of a speech, visual presentation, etc.) of an event such as a 
conference, by use of a keyword* The keyword is used to retrieve items of the 
event and create an event summary. In particular, a retrieving file is created based 
on the keyword A marker such as the word "determined" is attached to and 
identifies images or matters, allowing review of '^determined matters" without 
having to scan through other (i.e., non-marked) matters or images. 

The combination of Morag and Tada fails to teach the first element recited 
by claim 1 of "identifying, in response to a search query, first multimedia objects 
having an associated keyword that matches a keyword in the search query and 
second inultimedia objects that have content features similar to those of the first 
multimedia objects." 

The Examiner admits that Morag does not teach this recited element, and 
relies on the secondary reference Tada as teaching 'images retrieved by a retrieval 
keyword are displayed in the form of a rectangular parallelepiped for each relevant 
item" citing Tada at col, 10, lines 9-1 1 and "second set of images collected" citing 
Tada at col. 1 1, lines 21-23. The Examiner specifically states that **[t]his teaches 
the second set of images collected". 

Although Tada performs an initial keyword search which retrieves images, 
Tada does not describe ^identifying, in response to a search query . . . second 
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mxxltimedia objects that have content features similar to those of the first 
multiinedia objects" as recited in claim 1. The "second set of images collected" in 
Tada does not have content features similar to tfiose of the first images (objects). 

The image volume 22 described in Tada is filled by images retrieved by a 
keyword and is generated by collecting related images. Such "related images" are 
actually what Tada refers to as "deteraiined matters". Tada col. 11, lines 21-25. 
The determined matters described in Tada are searched by keyword- The 
determined matters (images) are marked with a word such as '"determined" and 
allow the possibility "to review the determined matters in a short time". Tada at 
col. 1 1, lines 1 1-20, Therefore the second set of images, the "related images" that 
the Examiner refers, are determined matters which are also retrieved through a 
keyword search, not through content features similar to the first images 
(multimedia objects). 

The combination of Morag and Tada fails to teach "monitoring feedback 
from the user as to which of the first and second multimedia objects are relevant to 
the search query^ and annotating one or more of the multimedia objects, which are 
deemed relevant by the user, with the keyword " 

The Examiner specifically states that "Morag teaches a mfcthod comprising 
monitoring feedback firom a user as to which of the first and second multimedia 
objects are relevant to the search query (col. 6, lines 53-55); and aimotating one or 
more of the multimedia objects, which are deemed relevant by the user, with 
keyword (col 9, lines 35-40), 

The "monitoring feedback fi^om a user" that is described in Morag actually 
occurs when an album is generated by a service provider for the user, not in 
response to a search query. ^The album may be generated based on the thumbnail 
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images, and the customer may comment on these images, during the upload of the 
complete image set* \ Morag col. 6, lines 53-55, Therefore Morag fails to teach or 
suggest the recited element of '^monitoring feedback from the user as to which of 
the first and second multimedia objects are relevant to the search query**. 

The annotation described in Morag is when a user acquires an image such 
as when a user takes a picture with a digital camera. Morag, coL 9, lines 36-37. 
The annotation becomes part of the image and may be used to group the image 
with other images. Morag does not disclose that annotation can take place after 
the images are presented in the albiim. The annotation described in Morag is not 
performed on objects that are retrieved based on a search query* Therefore Morag 
fails to teach or suggest the recited element of **annotating one or more of the 
multimedia objects, which are deemed relevant by the user, with the keyword". 

Accordingly, a combination of Morag and Tada fails to teach or suggest the 
claimed methods. Applicants respectfully request that the §103 rejection of claim 
1 be withdrawn. 

Dependent claims 6«9 ate allowable by virtue of their dependency on base 
claim L Applicants respectfully request that the §103 rejection of claims 2-9 be 
withdrawn. 

Independent claim 10 recites 

A metiiod comprising: 

iteratively retrieving multimedia objects from a database and 
monitoring feedback from a user as to whether the multimedia objects are 
relevant to a keyword in a search query; and 

annotating the multimedia objects based on the user's feedback, with 
the keyword. 
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The combination of Morag and Tada fails to teach or suggest the method of 
claim 10. The Examiner specifically states that **Morag teaches a method 
comprising: monitoring feedback from a user [as] to whether the multimedia 
objects are relevant to a keyword in a search query (col. 6, lines 53-55); and 
annotating the multimedia objects based on the user's feedback (coL 9» lines 35- 
4[0]r. 

As discussed above in support of claim 1, the "monitoring feedback from a 
usef ' described in Morag occurs when an album is generated by a service provider 
for the user, not in response to a search query. In addition, also discussed above in 
support of claim 1, the annotation described in Morag is when a user acquires an 
image, and is not based on the **user's feedback" where the user feedback is in 
response to a search query. 

The Examiner admits that Morag does not explicitly teach "iterativcly 
retrieving multimedia objects from a database" and relies on Tada at col. 10, lines 
21-23, The re-retrieval described by Tada; however, provides no assistance in 
light of Morag as to the recited method of claim 10. 

Accordingly, a combination of Morag and Tada fails to teach or suggest the 
claimed methods. Applicants respectfully request that die §103 rejection of claim 
10 be withdrawn. 

Dependent claims 11, 13-17 are allowable by virtue of their dependency on 
base claim 10. Applicants respectfrilly request that the §103 rejection of claims 1 1, 
13-17 be withdrawn. 

Independent claim 32 recites: 

A system comprising: 
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an infonnation retrieval unit to retrieve multimedia objects from a 
database based on a search query; 

a relevance feedback unit to capture a user*s feedback as to wliether 
the multimedia objects are relevant to ttie search query; and 

an annotation unit to amiotate, with a keyword, the multimedia 
objects based on the user's feedback. 

The combination of Morag and Tada fails to teach or suggest the method of 
claim 32. The Examiner presents the same arguments in rejecting claim 32, as 
those presented in rejecting claims 1 and 10, Applicants assert the arguments in 
support of claims I and 10. Applicants respectfully request that the §103 rejection 
of claim 32 be withdrawn. 

Dependent claims 33A2 are allowable by virtue of their dependency on 
base claim 32. Applicants respectfully request that the §103 rejection of claims 
33'42 be withdrawn. 

C^imJl 

Claim 12 is rejected under 35 U.S.C, §103(a) as being unpatentable over 
Morag in view of Tada, and further in view of U.S. Patent 5,579,471 to Baiber et 
al (Barber), Apphcants respectfully traverse the rejection. 

Claim 12 depends from claim 10 and hence incorporates the features of 
claim 10, As such claim 12 requires ''iteratively retrieving multimedia objects 
from a database and monitoring feedback from a user as to whether the multimedia 
objects are relevant to a keyword in a search query; and annotating the multimedia 
objects based on the user's feedback, with the keyword". 

Barber is cited for teaching "the searching method utilizing both keywords 
and features". Barber; however, provides no assistance in light of Morag and Tada 
as to the recited method of claim 10 from which claim 12 dq}ends. 



> atim 3:43:S1 PM [Eastern DayDprime] 



17 



.OCT 02 2003 12:47 FR LEE - HAYES PLL 509 323 8979 TO 17037467239 P. 21/25 



3 



5 



1 Applicants assert the arguments in support of claim 10. Applicants 

2 respectfully request that the §103 rejection of claim 12 be withdrawn. 
Claims 2-5 

Claims 2-5 are rejected under 35 U.S-C, §103(a) as being unpatentable over 
Morag in view of Tada, and further in view of US. Patent 5,899,999 to De Bonet 

6 (De Bonet). Applicants respectfully traverse the rejection. 

7 Claims 2-5 depend from claim 1 and hence incorporates the features of 

8 claim 1. As such claim 12 requires "identifying, in response to a search query, first 

9 multimedia objects having an associated keyword that matches a keyword in the 

10 search query and second multimedia objects that have content features similar to 

11 those of the first multimedia objects; presenting the first and second multimedia 

12 objects to a user; monitoring feedback from the user as to which of the first and 

13 second multimedia objects are relevant to the search query; and annotating one or 

14 more of the multimedia objects, which are deemed relevant by the user, with the 

15 keyword." 

16 De Bonet is cited for teaching "adjusting the weight"- De Bonet; however, 

17 provides no assistance in Ught of Morag and Tada as to the recited method of 

18 claim 1, from which claims 2-5 depend. 

19 Applicants assert the arguments in support of claim 1. AppUcants 

20 respectfully reqpiest that the § 103 rejection of claims 2-5 be withdrawn. 

21 Claims 18-24 

22 Claim 18-24 are rejected under 35 U.S.C. §103(a) as being unpatentable 

23 over U.S. Patent 6,314,420 to Lang et al. (Lang) in view of Morag. Applicants 

24 respectfully traveise the rejection. 
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Independent claim 18 recites 
A method comprising: 



retrieving multimedia objects according to a content-based retrieval 
4 process; 

^ presenting the multimedia objects to a user; 

^ monitoring feedback from the user as to \s1uch of the multimedia 

7 1 objects are relevant; and 

^ annotating one or more of the multimedia objects based on the user's 

feedback, with a keyword. 

The combination of Lang and Morag fails to teach or suggest the method of 
claim 18, The Examiner relies on Lang as teaching '^retrieving multimedia objects 
according to a content-based retrieval process; presenting the multimedia objects 
to a user, and monitoring feedback from the user as to which of the multimedia 
objects are relevant"; however, does not point where in Lang such methodology is 
taught or suggested. Nevertheless, the Examiner admits that Lang does not teach 
"annotating one or more of the multimedia objects based on the user's feedback, 
with a keyword" and relies on Morag. As discussed above in support of claims 1 
and 1 0, the annotation disclosed in Morag is performed by the user prior to entry in 
a searchable database, and prior to performing a search query. Applicants 
respectfully request that the §103 rejection of claim 18 be withdrawn. 

Dependent claims 19-24 are allowable by virtue of their dependency on 
base claim 18. Applicants respectfiilly request that the §103 rejection of claims 
19-24 be withdrawn. 
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Claims 25-31 

Claim 25-31 are rejected under 35 U.S,C. §103(a) as being unpatentable 

3 over De Bonet in view of Morag- 

4 Independent claim 25 recites 

A method comprising: 



9 
10 



13 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 



maintaining associations between keywords and multimedia objects, 
the associations being weighted to indicate how relevant the keywords are 
to the multimedia objects; 

retrieving a set of one or more multimedia objects for presentation to 

auser; 



monitoring feedback from the user as to wMch of the multimedia 
' ^ obj ects are relevant; and 



adjusting the weights of the associations based on the user's 
feedback. 

The combination of De Bonet and Morag fails to teach or suggest the 
method of claim 25. The Examiner reUes on De Bonet as teaching '^maintaining 
associations between keywords and multimedia objects, the associations being 
weighted to indicate how relevant the keywords arc to the multimedia objects; 
retrieving a set of one or more multimedia objects for presentation to a user, and 
adjusting the weights of the associations based on the user*s feedback " Not only 
does De Bonet fail to teach or suggest "adjusting the weights of the associations 
based on the user's feedback" but teaches away from feedback from a user, 
'^Advantageously, my invention renders the search and retrieval operations totally 
independent of the user, thus advantageously eliminating a need for the vscr to 
have any prior knowledge of any characteristic used to classify an image, let alone 



ReceivedfroiiK 509 323 8979 > atim 3:43:51 PM [Eastern DayOghtTime] 



20 



.OCT 02 2003 12:48 FR LEE - HAYES PLL 509 323 8979 TO 17037467239 



P. 24/25 



10 
It 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 



weight that characteristic vis-4-vis other such characteristics"- De Bonet at col. 8, 
line 64 to coL 9, line Z 

Morag is cited for teaching 'the feedbacks are given from the user"* 
Morag, however; provides no assistance in light of De Bonet as to the recited 
method of claims 25- Applicants respectfully request that the §103 rejection of 
claim 25 be withdrawn. 

Dependent claims 26-31 are allowable by virtue of their dependency on 
base claim 25. Applicants respectfully request that the §103 rejection of claims 
26-31 be withdrawn. 
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roNCLUSION 

All pending claims 1-42 are in condition for aUowance. Applicant 
respectfully requests reconsideration and prompt issuance of the subject 
application. If any issues remain tiiat prevent issuance of this application, the 
Examiner is urged to contact the undersigned attorney before issuing a subsequent 
Action. 
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Dated: ^U^^rZfJk. 



Respectfully Submitted, 



By: 



Emmanilel A, Rivera 

Reg. No. 45,760 

(509) 324-9256 ext 245 
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